Reconstruction of irregularly-sampled images by regularization in spline spaces

We are concerned with the reconstruction of a regularly-sampled image based on irregularly-spaced samples thereof. We propose a new iterative method based on a cubic spline representation of the image. An objective function taking into account the similarity to the known samples and the regularity of the function is minimized in order to obtain a good approximation. We apply the developed algorithm to motion-compensated image interpolation. Under motion compensation, the resulting sampling grids are irregular and require the irregular/regular interpolation. We show experimental results on real-world images and we compare our results with other methods proposed in the literature. 1. PROBLEM STATEMENT In image processing and coding, irregular sampling grids result from motion or disparity compensation, such as in motion-compensated video coding, motion-compensated video interpolation or disparity-compensated interpolation in stereoscopic images. In all these applications, when a vector field describing the transformation intercepts an image plane of interest, the resulting grid is, in general, irregular (nonuniform) and a method to recover a regularly-sampled image from irregularly-spaced samples is needed. Many methods varying from polynomial interpolation or approximation found in computer graphics to algorithms from the theory of signals have been proposed and applied to image reconstruction from irregularly-spaced samples in some special cases [1, 2]. However, there is no general algorithm that fits all applications and the subject is still open to research. In this paper, we are concerned with irregular sampling resulting from motion or disparity compensation. Let be a sampled (discrete) image defined over a regular 2-D sampling grid ! "$#% '& . Let () * +, .0/1 2.034 5" # 6 7 be a vector field describing a transformation (motion or disparity compensation) that, applied to image results in transformed image 8 . An irregular sampling grid 9 : <;!=> * @? AB+, 5" DC = EGF is obtained by intercepting the vector field with an image plane parallel to that of the image and separated by distance A . Samples H of the transformed image 8 are obtained by assigning intensity values from image at motion-compensated positions, i.e., H I J H ; = K L M N 2 6 ; => 9 $OP Q& . The goal is to develop a reconstruction algorithm in order to recover the transformed image 8 sampled over regular grid from the irregularly-sampled image H .